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Hypothesis Testing Minority Game (HMG) is a variant of the standard Minority Game (MG) 
that models the inertial behavior of agents in the market. In the earlier study of our group, we find 
that agents cooperate better in HMG than in the standard MG when strategies are picked from 
the full strategy space. Here we continue to study the behavior of HMG when strategies are chosen 
from the maximal reduced strategy space. Surprisingly, we find that, unlike the standard MG, the 
level of cooperation in HMG depends strongly on the strategy space used. In addition, a novel 
intermittency dynamics is also observed in the minority choice time series in a certain parameter 
range in which the orderly phases are characterized by a variety of periodic dynamics. Remarkably, 
all these findings can be explained by the crowd-anticrowd theory. 
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I. INTRODUCTION 

Studying economic systems by agents-based models 
have attracted the attention among physicists in recent 
years [H, 0, OJ. O ne of the most famous agents-based 
model in this regard is the Minority Game (MG) 
MG does not only capture the fact that all people in the 
market think inductively and selfishly [5j, its complex- 
ity also satisfy the definition of a complex system in the 
strictest sense @ . In spite of its simple governing rules, 
agents in this model self-organize giving rise to an unex- 
pected global cooperative phenomena. 

Using the standard MG as blue print, various modifi- 
cations to the rules of the standard MG have been pro- 
posed @, 0, 0, [HI to understand different aspects and 
phenomena in realistic economic systems. In particu- 
lar, Man and Chau introduced the Hypothesis Testing 
Minority Game (HMG) to model the inertial behavior 
of agents They found that the presence of inertial 
agents improve global cooperation leading to a decrease 
of the variance per agent over the entire parameter space 
provided that the strategies of each agent are chosen from 
the so-called full strategy space (FSS) [Til. H3|. 

In this paper, we move on to study the agent coopera- 
tion and the dynamics of HMG in case the strategies are 
picked from the so-called maximal reduced strategy space 
(MRSS) [llj]. We begin by briefly reviewing the rules of 
HMG and stating the parameters used in our numerical 
simulations in Sec. [ill Then we report our simulation 
results in Sec. [nil To our surprise, the behavior of HMG 
depends strongly on the strategy spaces used. Specifi- 
cally, agents generally cooperate better when strategies 
are picked from the FSS rather than the MRSS provided 
that they are sufficiently reluctant to change their strate- 
gies. In contrast, the standard MG is so robust that its 
dynamics and cooperative behavior are essentially the 
same irrespective of whether the FSS or the MRSS is 
used. Furthermore, we find that in HMG the minority 



choice time series exhibits intermittency in which the or- 
derly phases show periodic dynamics with period up to 
2(2 M — 1) whenever the memory size of the strategies 
M is greater than 1 in a certain parameter range when 
strategies are picked from the MRSS. This novel inter- 
mittent phenomenon does not show up in HMG provided 
that strategies are picked from the FSS as well as in the 
standard MG. We explain how these differences originate 
from the choice of the strategy space by a semi-analytical 
approach known as the crowd-anticrowd theory [Til . Il5j 
in Sec. lIVl In fact, the major reason responsible for these 
differences is that it is a lot easier for an agent to keep on 
using one's currently adopted strategy when the strategy 
pool is the MRSS than rather than FSS in certain pa- 
rameter regime. Finally, we summarize our findings in 
Sec. |V1 Our findings show that extra care is needed to 
study variants of MG as their behavior may depend sen- 
sitively on the strategy space employed. Nonetheless, the 
ability to explain the behavior of HMG using the crowd- 
anticrowd theory suggests that this theory may still be 
useful to explain the dynamics of variants of the standard 
MG provided that one carries out the analysis carefully. 



II. HYPOTHESIS TESTING MINORITY GAME 

Recall that in the standard MG, agents act according 
to the predictions of their best performing strategies. In 
other words, agents in the standard MG do not hesitate 
to stop using their current strategies once the perfor- 
mance indicator, known as virtual score, shows that the 
strategies are not the best. In contrast, the HMG incor- 
porated the inertial behavior of agents by allowing them 
to stick to their currently using strategies until their per- 
formances are too poor to be acceptable. More precisely, 
a fixed real number Ik between 0.5 and 1.0 is assigned 
once and for all to each agent k in HMG to represent 
their reluctance to switch strategies. Using the value of 
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Ik as an indicator of the confidence level, agent k tests 
the hypothesis that his currently using strategy is his best 
strategy at hand at each turn. Furthermore, he switches 
to another strategy and resets the virtual scores of all his 
strategies to if the null hypothesis is rejected. Apart 
from these differences, the governing rules of HMG are 
identical to those of the standard MG. 

We state the rules of HMG below for reader's conve- 
nience. 



A. Rules of the game 

1. HMG is a repeated game of a fixed population of 
N agents. A number I k <E [0.5, 1) is assigned to 
agent k once and for all to represent his inertia. 

2. At each turn r, every agent has to make a choice 
between one of the two sides (namely side and 
side 1) based on the strategies to be described in 
rulcUJ Those agents in the side with the least num- 
ber of agents (known as the minority side) win in 
that turn. And in case of a tie, the winning side is 
randomly selected. 

3. The only piece of global information reveals to the 
agents at time r is the winning sides in the last M 
turns known as the history /I(t). 

4. Before the game commences, each agent k is as- 
signed once and for all S randomly picked strate- 
gies Sk,i for i = 0, 1, . . . , S — 1. Each strategy Sk,i 
is a function map from the set of all possible histo- 
ries to the set {0, 1} and its virtual score Qk,i is set 
to initially. Without loss of generality, strategy 
Sk,o is assumed to be the currently using strategy 
of agent k at the beginning of game. 

5. Agent k will switch his current strategy from Sk,o 
to Sk,j if and only if the maximum virtual score 
difference ASlj, drops below the threshold Xk\J2Tk, 
that is, 

Afl k = max{^ fe .o - Qk,i} = &k,o - ^k.j 

i 

< x k y/2r% (1) 
where x k is defined by 

— / e-*/*dx = I k (2) 
\/ in J Xk 

and Tfc is the number of turns elapsed since agent fc's 
last switch of strategy. In case agent k switches his 
strategy, he exchanges the labels and j so that his 
currently using strategy is always labeled as Sk.o- 
In addition, the virtual scores £lk,i are reset to 
for all i and Tk is reset to 1. 



6. Agent k uses his current strategy to guess the mi- 
nority choice of the current turn. Moreover, the vir- 
tual score of strategy Sk,i is increased (decreased) 
by 1 if it predicts the minority side of that turn 
correctly (incorrectly) . 

B. Parameters used in our simulation 

We select the following parameters in our simulations: 

1. TV is odd; 

2. S = 2; 

3. all values of Ik are chosen to be the same indepen- 
dent of the agent label fc (and we write this common 
Ik as / for simplicity); and 

4. Skj are picked from the so-called MRSS [l3|. (That 
is, Skj can be written in the form 

M 

Sr = VO + '^2rji^T-i , (3) 

i=l 

where s T is the prediction of the minority side in the 
Tth turn, T70, f?i, • • • , f?M £ {0, 1}, \ii is the minority 
side in the ith turn and the arithmetic is performed 
in the finite field of two elements GF{2) @, [H, 
fl6j . In other words, a strategy in the MRSS is 
characterized by (770, 771, . . . , t]m)-) 

With the exception of point |H the parameters used in 
this study are identical to those used in our earlier study 
of HMG reported in Refs. [Ill EH ■ in contrast, strategies 
in Refs. [O, [l2j are picked from the FSS, namely, the 
set of all possible strategies. And a strategy in the FSS 
may not be expressed as a linear function of /Vs. From 
now on, we use the symbols HMG FSS and HMG MRSS to 
denote the HMG in which strategies are picked from the 
FSS and the MRSS, respectively. 

III. OUR NUMERICAL SIMULATION RESULTS 

A. Focus of our study 

We are interested in both the cooperative behaviors 
and the dynamics of the game. Recall that MG and 
HMG are non-positive sum games in the sense that the 
number of winning agents is less than or equal to the 
number of losing agents in each turn. And we say that 
the agents (or the system) cooperate better if the av- 
erage number of winning agents per turn is high. Our 
numerical simulations show that agents self-organize in 
such a way that there is no bias in picking the minority 
side when averaged over the agents and the number of 
turns, so we follow the usual practice to study agent co- 
operation by means of the a = 2 M+1 /NS against a 2 /N 
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graph where a 2 is the variance of the number of agents 
choosing side [Hj]. The lower the value of a 2 /N, the 
better the agent cooperation. 

As for the dynamics of HMG, our investigation focuses 
on the analysis of the periodicity of the minority choice 
time series through the auto-correlation function. And 
the auto-correlation function can be conveniently stud- 
ied by means of a time lag t against auto-correlation Co 
graph. In order to make sure that the dynamics is gen- 
uine and long lasting, we only consider the time series 
after the system has equilibrated. We also perform sim- 
ulations using different values of N and initial quenched 
disorders to make sure that the dynamics we are going 
to report below arc generic. 

Actually the dynamics depends on the following three 
factors: 

1. number of agents ./V; 

2. history size M; and 
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3. the initial quenched disorder as reflected by the 
value / and the strategies Sk,i assigned to the 
agents. 



FIG. 2: The auto-correlation for HMG FSS with a = 6.4 x 
1(T 4 < a™ G when (a) I = 0.53 and (b) / = 0.90. 



Our choice of parameters for the HMG MRSS reported in 
Sec. Ill B I makes the dynamics of the game deterministic 
and hence enabling us to study the periodic dynamics 
of the minority choice time series easily. In contrast, 
when played using other choices of N and S, the non- 
deterministic nature of this game weakens the periodic 
dynamics in minority choice time series, making both 
the numerical and analytical studies more troublesome. 

Unlike the standard MG, we find that both the co- 
operative behavior and the dynamics of HMG depend 
strongly on the strategy space chosen. We shall elabo- 
rate more on this point in the coming two subsections. 
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FIG. 1: The a against a 2 /N graph for HMG FSS . The value 
of a is varied by fixing M and changing N. 



B. Reviewing the simulation results of the 



Recall from the earlier study of our group in Refs. |ll|, 
[IH that when I is chosen to be less than I Cl , where 

7 Cl = ^/ e- x2 / 2 dx, (4) 

V27T J- A /2/2 M + 1 

the inertia of agents is not strong enough to make the dy- 
namics of the HMG FSS to deviate significantly from that 
of the standard MG. Thus, the a 2 /N is about the same 
as that of the standard MG. Moreover, the well-known 
period 2 M+1 dynamics in the minority choice time series 
that appears in the standard MG when a is less than 
a MG o 3^ cr itical value for the standard MG, is 
also present here [Tll.fT^. In contrast, when I > I Cl , the 
inertia of agents becomes strong enough to significantly 
reduce the herd effect amongst agents resulting in a much 
lower a 2 /N (and hence indicating that agents cooperate 
better). Besides, the period 2 M+1 dynamics is no longer 
present when a < a^ G [ll|, [l^] ■ These earlier findings 
are summarized in Figs. [1] and [21 



C. Simulation results of the HMG Mftbb 

Contrary to our expectation, we find that the behav- 
ior of HMG MRSS is significantly different from that of 
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HMG FSS when I > I Cl and a < 1. The details of our 
findings are listed below. 

• I < I Cl : By comparing Fig. [1] with Fig. [3]a, and 
Fig.[2]with Figs.[4]and[5j we know that the behavior 
of HMG MRSS in this regime is similar to that of 
HMG FSS . That is, they have about the same level 
of agent cooperation. In addition, the standard MG 
El El, HMG FSS [HI and HMG MRSS all exhibit 
the same period 2 M+1 dynamics in the minority 
choice time series whenever a < a^ 1 and show no 
periodic dynamics for a a^ 1 . 

• I > I Cl and a > 1(> o^ G ): By comparing Fig. [1] 
with Fig. [31 we find that the values of a 2 /N are 
about the same for both HMG FSS and HMG MRSS 
in this re gim e. Moreover, the standard MG [Hj], 
HMG FSS 03 and HMG MRSS all show no periodic 
dynamics. 

• I > I C1 and a < 0.1(< af G ): Figs. Q] and [3] show 
that for the same value of a in this regime, the 
cooperation amongst agents for the standard MG 
is the worst, for HMG MRSS is in the middle and 
for HMG is the best. One interesting feature 
for HMG MRSS is that, unlike HMG FSS , the value 
of a 2 /N increases as a decreases in this regime in- 
dicating that agents cooperate less and less as the 
number of agents ./V increases (and with M and S 
held fixed). As for the dynamics, Fig.[5]depicts that 
the system exhibits no obvious periodic dynamics. 
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FIG. 3: Plots of a against a 2 /N for HMG MRSS with (a) / = 
0.53, 0.90 and (b) I = 0.95. 



Nonetheless, its minority choice time series condi- 
tioned on an arbitrary but fixed history exhibits a 
very weak period two dynamics. 

• I > I Cl and 0.1 < a w a^ G < 1: In this regime, 
we find that the value of a 2 /N obtained after equi- 
libration depends on the initial quenched disorder 
of the system indicating the presence of a phase 
transition point. (See Fig. [3l) Actually, the val- 
ues of a 2 /N obtained in many runs are rather close 
to the theoretical minimum of 1/4JV (which is at- 
tained when there are exactly (N — l)/2 winning 
agents in each turn) implying that agents cooperate 
almost perfectly. 

The dynamics of the minority choice time series in 
this regime is rather complex. Actually, no obvi- 
ous periodic dynamics is observed for those initial 
quenched disorder that ends up with values of a 2 /N 
about the same as those for I < I Cl . (See Fig. [7^.) 
In contrast, those ending up with a much smaller 
a 2 /N show intermittency. (See Fig. \7jp.) That is 
to say, when a 2 /N is small, the time series exhibits 
periodic dynamics for some time and then the pe- 
riodicity either suddenly disappears or the period 
of the dynamics changes. Also the brief episode 
of aperiodicity terminates with the commencement 
of a new periodic dynamics (with possibly a new 
period). 

Interestingly, the period of the orderly phase for 
this intermittency depends on the value of /. In 
case I Cl < I < I C2 , where 

i f + OO 

I C2 = —= \ e~ x I 2 dx « 0.92135 , (5) 
V2vr 

the periods are less than or equal to 2(2 M — 1) when- 
ever M > 2. More importantly, these periods are in 
the form 2 3 L.C.M.(pi,p 2 , ■ • ■ ,Pm) where j > 1 is an 
integer, L.C.M. denotes the least common multiple 
of the M arguments and pi, ■ ■ ■ ,Pm are positive in- 
tegers dividing (2 M — 1). Clearly, this phenomenon 
is novel and is never found in both the standard MG 
and HMG FSS . We observe the trend that long pe- 
riod dynamics tends to be more stable in the sense 
that it lasts longer. In fact, the longest periodic 
dynamics, namely the one with period 2(2 M — 1), 
appears to be the most stable. However, being the 
most stable dynamics does not necessarily mean 
that it must show up for every initial quenched dis- 
order. Actually, the period 2(2 M — 1) dynamics is 
harder and harder to find as M increases beyond 
about 7. 

In the case of I > I C2 , the periodic dynamics of 
the orderly phase of the intermittency is weak com- 
pared with the case of I Cl < I < I C2 . As shown 
in Fig. [7J;, the maximum period is in the form 
2 J (2 — 1) where j is an integer greater than or 
equal to 2. 



Our findings of the dynamics of the minority choice 
time series can be tabulated in Table HI 



Period of 
dynamics 


MG 


HMG FSS 


HMG MRSS 


i < r 1 


i > r 1 




I>I C1 


a > a c 


nil 


nil 


nil 


nil 


nil 


MG 

a « a c 


2 M+1 


nM+l 


nil 


2M+1 


intermittent" 


„ " „MG 


2 M+1 




nil 


2M+1 


nil 6 



"The maximum period of the orderly phase of this dynamics is 
23 (2 M - 1) where the value of j can be found in the main text. 

'But the minority choice time series conditioned on an arbitrary 
but fixed history shows a very weak period two dynamics 

TABLE I: Summary of the dynamics in the minority choice 
time series for MG and HMG for odd number of agents and 
M > 2. 
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FIG. 4: The auto-correlation of HMG MRSS for a = 0.63 > 
o^ G when (a) I = 0.53 and (b) I = 0.90. 



D. Conditions for the period 
2 J L.C.M.(pi,p2, • ■ • ,Pm) dynamics in the orderly 
phase of the HMG MRSS intermittency 

We find that in HMG , some agents seldom change 
their strategies while others do so frequently. We say that 
an agent is oscillating if he switches strategy within the 
previous 2 M+1 turns. Otherwise, the agent is said to be 
frozen. It turns out that the number of frozen agents 
and their performance provide important information to 
allow us to understand the origin of the dynamics in the 
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FIG. 5: The auto-correlation of HMG MRSS for I < F 1 and 
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FIG. 6: The auto-correlation of HMG MRSS when a = 6.4 x 
10" 4 < af G together with (a) I C1 < I < I C2 and (b) I > I c ' 2 . 



minority choice time series of HMG MRSS for I > I Cl and 
a « af G . 

Table |TT] shows the average probabilities for a frozen 
(an oscillating) agent to correctly predict the minority 
side in a typical HMG MRSS with I > I C1 and a w a^ G . 
Clearly, a frozen (an oscillating) agent will have a higher 
(lower) chance to correctly predict the minority side (that 
is, the winning probability) in the next turn. Since an os- 
cillating agent must lose frequently in recent turns, our 
finding means that badly performing agents are likely 
to perform badly in future. More importantly, numer- 
ical simulations tell us that the presence of the period 
2 J L.C.M.(pi,p2, • • ■ iPm) dynamics in the orderly phase 
of the intermittency where j > 1 is an integer and 
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FIG. 7: The auto-correlation of HMG MRbb for a = 0.064 « 
c^ G with (a) I C1 < I < J C2 and cr 2 /^ is large, (b) I C1 < I < 
F 2 and a 2 /N is small, and (c) I > F 2 and cr 2 /7V is small. 



AT 


mean winning probability 


frozen agents 


oscillating agents 


4001 


0.499 


0.292 


8001 


0.499 


0.302 


16001 


0.499 


0.293 



TABLE II: The winning probability of frozen and oscillating 
agents for M = 3 and / = 0.90 averaged over 100 independent 
runs. 

pi,...,PM divide (2 M — 1) is almost always accompa- 
nied by the presence of only one oscillating agent in the 
entire system. Besides, this oscillating agent uses only 
two distinct strategies that are anti-correlated. (That is, 
these two strategiespredict different minority side for 
all given histories [3, EH-) Moreover, the strategies 
used by the remaining frozen agents form a collection 



of crowd-anticrowd pairs. Interestingly, our observed in- 
termittency disappears and becomes a single aperiodic 
phase if we replace the histories by random variables or 
if the number of agents N is even. Also, the sole oscillat- 
ing agent associated with each orderly phase of the time 
series may be different. 



IV. THE CROWD-ANTICROWD 
EXPLANATION 

Let us briefly review how the crowd-anticrowd theory 
explains the behavior of the standard MG before adapt- 
ing it to explain the behavior of HMG FSS and HMG MRSS . 

A. The crowd-anticrowd theory for the standard 
MG 

According to the crowd-anticrowd theory, agent coop- 
eration in the standard MG is determined by the number 
of (effective) anti-correlated pairs of strategies in current 
use. The smaller the difference between the number of 
agents currently adopting a strategy a and those cur- 
rently adopting its anti-correlated strategy a, the bet- 
ter the crowd-anticrowd cancellation leading to a bet- 
ter agent cooperation. Since the number of available 
strategies is much less than the strategy space size for 
a a™ , crowd-anticrowd cancellation cannot be effec- 
tive in this regime. And because standard MG agents do 
not have inertia, they switch strategies immediately once 
the maximum virtual score difference is negative. Due to 
the fact that the virtual score of a strategy in the stan- 
dard MG is independent of who owns or uses it, every 
standard MG agent has the same view on the perfor- 
mance of a given strategy. So, when a <C a^ G , standard 
MG agents tend to adopt and drop similar strategies all 
the time. This over-reaction leads to a herd effect and is 
the origin of the maladaptation in the standard MG in 
this regime. Thus, effective agent cooperation is possible 
only for a w a^f G in the standard MG. A remarkable 
feature of the standard MG is that effective agent coop- 
eration is indeed possible in this regime in spite of the 
fact that agents act independently by utilizing common 
global coarse-grained information only [3, EBl • 

B. Towards the crowd-anticrowd theory for HMG 

Unlike the standard MG, HMG agents use Eq. flU 
to decide whether to keep their currently using strate- 
gies or not. In general, they are initially assigned dif- 
ferent strategies and begin to adopt their currently us- 
ing strategies at different times. So, they have different 
virtual score difference Af2fc and the number of turns 
since the adoption of the current strategy Tfc. Together 
with the virtual score reset mechanism stated in rule [3] 
of HMG, the same strategy may be ranked differently 
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amongst HMG agents. Consequently, by picking strate- 
gies from the same strategy space, the effective strategy 
diversity for HMG is in general bigger than that for the 
standard MG. Furthermore, the higher the value of / 
(and hence the higher the inertia) , the slower the average 
rate of strategy switching. All these factors reduce HMG 
agents' over-reaction and the herd effect making the sys- 
tem to better cooperate [111, [l2| . Just like the standard 
MG, the dynamics of HMG encourages agents to form 
crowd- anticrowd pairs thereby increasing agent coopera- 
tion. But unlike the standard MG, this "crowd-anticrowd 
pair formation" driving force in HMG is more gentle and 
is less likely to cause over-reaction and maladaptation 
because of the presence of inertia. Hence, the higher the 
inertia, the longer the equilibration time. At this point, 
we have to emphasize that the presence of inertia need 
not imply that the agents must cooperate because the 
"crowd-anticrowd pair formation" driving force may not 
be strong enough in a certain parameter regime. The 
bottom line is that the presence of inertia never worsen 
the agent cooperation. In other words, for fixed values of 
M , N and S, the variance of attendance per agent a 2 /N 
for the standard MG has to be greater than or equal to 
that for the HMG FSS or HMG MRSS . 

Suppose the agents really cooperate. Still there are two 
ways to prevent them from cooperating forever. Con- 
sider a history fl which has non-zero probability of oc- 
currence. Suppose further that the minority choice time 
series conditioned on this history /2 is biased. (That is, it 
is more likely to find a particular minority choice than the 
other in this conditioned time series.) In this case, cer- 
tain strategy a will outperform its anti-correlated partner 
a in the long run. More precisely, the rate of change of 
virtual score difference between a and a averaged over a 
sufficiently long number of turns is positive. So, after a 
sufficiently long time, agents will begin to drop strategy 
a and adopt strategy a, making strategy a much more 
popular than strategy a. In particular, if the minority 
choice time series exhibits periodic dynamics, then the 
timescale for a frozen agent to change to an oscillating 
agent via this mechanism is directly proportional to the 
period of the dynamics and inversely proportional to the 
number of biased histories in the minority choice time 
series. 

Even if the minority choice time series conditioned on 
every history is un-biased, there is still a way for agents 
to stop cooperating. Provided that the value of Aflk 
follows an un-biased random walk, after sufficiently long 
time, agent k can switch his strategy once a while due 
to fluctuations in Af2&. These two mechanisms act like 
a "crowd-anticrowd pair destruction" driving force that 
decreases agent cooperation. Surely, the former mecha- 
nism is more efficient. 

In summary, it is the combined actions of the above 
two driving forces that determine the agent cooperation 
and dynamics of HMG. In fact, the "crowd-anticrowd 
pair formation" driving force dominates the initial dy- 
namics of the HMG. And the "crowd-anticrowd pair de- 



struction" driving force becomes important after most of 
the agents have been paired up. This picture allows us 
to understand the simulation results reported in Sec. IIHI 



C. The crowd-anticrowd explanation for 



In the case of a > > there are so few strate- 
gies at play that most of the crowd-anticrowd pairs is 
made up of only one agent. As a result, the "crowd- 
anticrowd pair formation" driving force is never strong 
enough to ensure agent cooperation irrespective of the 
value of /. Thus, crowd-anticrowd cancellation is ineffec- 
tive. Besides, agents in effect make random choices each 
turn so that the value of a 2 /N approaches the coin-toss 
limit as a — > oo. Surely, the minority choice time se- 
ries does not show any periodic dynamics [Til . [l5| . Since 
the above arguments are also valid for the standard MG 
and HMG FSS , we understand why MG, HMG FSS and 
HMG MRSS all behave in the same way in this parameter 
regime. 

In the case of I ;$ I Cl , the inertia of agents is so low 
that agents switch strategies immediately whenever the 
maximum virtual score difference is negative. In other 
words, the response of standard MG and HMG agents are 
the same in this parameter regime Hence, the agent 
cooperation and dynamics of standard MG, HMG FSS and 
hmg mrss are a b out tn e same. 

The remaining case to study is / > I Cl and a < ajr . 
Note that the behavior of HMG FSS and HMG MRSS in this 
case differ markedly as reported in Sec. lIIII We begin our 
analysis by stating the following claim whose proof can 
be found in Appendix [A"l 

Claim 1 Let N be an even number. Suppose each of the 
N players are randomly and independently assigned S 
socks; and a sock has 2 M possible colors and can either 
be left or right. (Hence, there are 2 M+1 kinds of socks.) 
Suppose further that each kind of sock is selected with 
equal probability. Then, provided that 2 M+1 < NS, the 
probability that there exists a way to form N/2 pairs of 
socks by picking exactly one sock from each of the N play- 
ers is greater than or equal to 1 — (3/N for some positive 
(3 which is independent of N. 

Recall that two strategies are said to be anti-correlated 
if they always predict different minority side. And they 
are said to be uncorrelated if they have equal chance to 
predict the same minority side provided that each history 
occurs with equal probability. Thus, strategies in the 
MRSS consist of 2 M pairs of anti-correlated strategies 
and that str ateg ies picked from two distinct pairs are 
uncorrelated [l4flH. 

By identifying the 2 M+1 different kinds of socks with 
the 2 M+1 different strategies in the MRSS, Claim [Dim- 
plies that for a sufficiently large N and for a < 1, 
the probability of forming [-^/2J pairs of anti-correlated 
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strategies by picking one strategy from each agent's strat- 
egy pool is high. Surely, finding this solution requires 
communications amongst the agents. If the agents could 
keep on using his particular choice of strategy throughout 
the game, maximum agent cooperation would be attained 
and hence the theoretical minimum value of a 2 /N (that 
is a 2 /N — if N is even and 1/4 AT if TV is odd) would be 
resulted. In contrast, since the strategy space size of the 
FSS is exponentially larger than that of the MRSS in the 
large M limit, the condition a < 1 is not sufficient for 
agents to maximally cooperate in the above way should 
they pick strategies from the FSS. The correct condition 
in this case should be 2 2 < NS (and hence, a — > + in 
the thermodynamic limit). 

Claim [JJ only assures the existence of an optimal way 
of agent cooperation with a high probability. It remains 
to show that this maximum agent cooperation can be 
achieved with a high chance for HMG MRSS under certain 
conditions. Recall from step [5] of the rules of HMG that 
agent k uses the maximum virtual score difference Afi^ 
amongst all the strategies initially assigned to him to 
decide whether to change strategy or not. The following 
consequences of step [5] are important to understand the 
strong dependence of HMG on the strategy space used: 

1. The higher the value of I/., the more willingly for 
agent k to stick to his currently using strategy. 

2. For fixed values of Ij-'s, agents with a strategy and 
its anti-correlated partner in their pool of strategies 
have the strongest tendency, on average, to switch 
their strategies. 

3. Suppose agent k has just switched to a new strat- 
egy and that this newly adopted strategy incor- 
rectly predicts the minority side in its first use, then 
Tfc = 1 and Afifc = —2. As a result, this agent will 
drop his newly adopted strategy in the next turn 
provided that Ik < I C2 ■ 

1. The sub-case of I > I C1 and a ps af 

Shortly after the commencement of HMG MRSS , the mi- 
nority choice time series should resemble an un-biased 
random sequence. So provided that I is sufficiently large, 
Subsec. TlV Bl tells us that the "crowd-anticrowd pair for- 
mation" driving force allows a large number of agents 
to form crowd-anticrowd pairs. Most of these paired 
agents will be frozen, and there are only a few oscillat- 
ing agents in the system. By simple probability consid- 
eration, we expect that most of the strategies hold by 
these oscillating agents are un-correlated. As the dy- 
namics of the system, which is determined mostly by the 
dynamics of these oscillating agents, the minority choice 
time series conditioned on an arbitrary history is likely to 
be un-biased. Hence, fewer and fewer oscillating agents 
will present as they gradually form crowd-anticrowd pair 
and become frozen. From Claim [TJ we believe that for 



a sufficiently large /, agents in HMG MRSS have a high 
chance to attain maximum agent cooperation provided 
that a 1=3 a™ < 1. This is consistent with the findings 
in our numerical simulations reported earlier in Sec. IIIII 
that the highest chance of finding maximum agent co- 
operation is when a o^ G and / > I Cl . And this 
maximum agent cooperation is accompanied by the ex- 
istence of at most one oscillating agent in the system 
who switches between a pair of anti-correlated strategies. 
This finding agrees with the discussion following Claim [1] 
that the agent holding a pair of anti-correlated strategies 
switches his strategy most readily. To conclude, when 
N is odd, the effective number of strategies at play for 
HMG MRSS in this regime is reduced to one in most of 
the time. Thus, the frozen agents have an average of 1/2 
chance to correctly predict the minority side in the next 
turn while the only oscillating agent has no chance to do 
so. Consequently, the frozen agents are unlikely to switch 
their currently using strategies while the only oscillating 
agent is prone to strategy switching. 

We now discuss the dynamics of the minority choice 
time series when A?" is odd. From the above discussions, 
it is clear that the system is in an orderly (chaotic) phase 
whenever it has one (more than one) oscillating agent. In 
addition, this oscillating agent is most likely to be switch- 
ing between a pair of anti-correlated strategies. This os- 
cillating agent always predicts the minority side incor- 
rectly throughout the corresponding orderly periodic dy- 
namics phase. Thus, from the discussions immediately 
after Claim [TJ we conclude that this oscillating agent 
must drop his currently using strategy and switch to its 
anti-correlated counterpart in each turn provided that 
I Cl < I < I C2 . By identifying a strategy with a linear 
function of the historical minority choices \ii 's over the fi- 
nite field GF(2) in the form of Eq. ([3]), we know that the 
difference between the linear functions associated with 
two anti-correlated strategies is equal to 1. So, whenever 
a w ct^ G and I Cl < / < I C2 , the minority side in the nth 
turn /!„ throughout this orderly periodic dynamics phase 
obeys 

M 

Mn = X! Vif^-i + Vo + n , (6) 
»=i 

where r)i, fa G GF(2) and n denotes 1 + 1 + ••• + 
1 (n terms). 

We show in Appendix [B] the following theorem. 

Theorem 1 The sequence {// n } generated by the recur- 
sion relation in Eq. {SJ) is periodic. Its period is in 
the form 2 J L.C.M.(pi,p2, ■ ■ ■ ,Pm) where j > 1, and 
pi, . . . ,pm are positive integers dividing (2 M — 1). More- 
over, the longest possible period for this sequence is 
2(2 M - 1) if M > 2 and 4 if M = 1. Suppose the 
sequence is of maximum possible period and M > 2. 
Denote the history (fin-M+u fin-M+2, • > ■ , fbi-u Hn) by 
ft(n) . Then, the two histories of alternating 's and 1 's, 
namely, (0, 1, 0, 1, . . .) and (1, 0, 1,0,.. .) appear in the se- 

quence of history H — {/Lt(n)} n=1 once; while all the 
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other (2 M — 2) possible histories all appear in the history 
sequence H twice. 

This theorem allows us to explain the period of the or- 
derly phase of the time series in the case of I Cl < I < I C2 
and a w a^ G - In particular, the proof tells us that 
the longest periodic dynamics in the orderly phase for 
M > 2 is of period 2(2 M — 1). Nevertheless, periods sat- 
isfying Theorem [TJ do not show up in the orderly phases 
equally frequently. If the number of turns between two 
consecutive occurrence of a history in an orderly phase 
is odd, then the minority time series conditioned on this 
history shows period two dynamics because the sole oscil- 
lating agent makes alternating prediction of the minority 
side each time when he is given the same history. In 
other words, the minority choice time series conditioned 
on this history is un-biased. In contrast, if the number 
of turns between two consecutive occurrence of a history 
in an orderly phase is even, the minority time series con- 
ditioned on this history is biased for it exhibits period 
one dynamics. Thus, the "crowd-anticrowd destruction" 
driving force discussed in Subsec. IIVBI will break the 
maximum agent cooperation in a time proportional to 
the period of this orderly phase and inversely propor- 
tional to the number of conditional minority choice time 
series that exhibits period one dynamics. 

Theorem [1] tells us that for M > 2, the longest possible 
period of the orderly phase is 2(2 M — 1). This dynamics 
is present when the (degree M) characteristic polynomial 
of the recursion relation ©, which is associated with the 
pair of anti-correlated strategies used by the sole oscil- 
lating agent, is primitive. In a single period, the two 
histories that consists of alternating O's and l's appear 
once while all other histories appear twice. Therefore, 
the number of turns between two consecutive appear- 
ance of each of the two histories consisting of alternating 
O's and l's equal the even number 2(2 M - 1). It is easy 
to show that the number of turns between two consecu- 
tive appearance of all other histories must be odd. (One 
way to do so is that if /x(fc) = /x(fc') with (k — k') being 
a positive even number, the homogeneous parts of the 
solutions of Eq. ([6]) for n = k — i and n = k' — i are 
equal whenever i = 0,1,.. . ,M — 1. And this is possi- 
ble only when (2 M - 1) | (k - k'). As (k - k') is even, 
so k — k' > 2(2 M — 1). Since all other histories occurs 
twice in a single period, the number of turns between 
two consecutive occurrence for them must be odd.) In 
conclusion, the 2(2 M — 1) period dynamics can be found 
in the minority choice time series although it cannot be 
ever-lasting. Since out of the 2 M possible minority choice 
time series conditional on a particular history, only two 
of them show period one rather than period two dynam- 
ics, the period 2(2 M — 1) dynamics in the minority choice 
time series is quite stable in the sense that it lasts for a 
longer time. This also explains why the stability of this 
period increases with M. 

As M increases while keeping S and a fixed, the prob- 
ability of having an agent in the system that holds a 
pair of anti-correlated strategy is about 1 — [1 — S(S + 



l)/2 M+2 ] N 1 - e -(s+i)/2«. As a « af G w 0.3 in 
the intermittent phase, it is not surprising that there is a 
high chance for the sole oscillating agent in this phase to 
switch between two anti-correlated strategies. Nonethe- 
less, from the proof of Theorem [TJ the corresponding 
probability of having an agent in the system that holds 
a pair of anti-correlated strategy that causes the period 
2(2 M - 1) dynamics is about 1 - [1 - S(S + l)(p(2 M - 
1)/2 2 ( M+1 )M] N , where <p denotes the Euler-Totient func- 
tion. By prime number theorem, this probability is equal 
to at least 1 - [1 - S(S + 1)(2 M - i)/2 2 ( M + 1 )M c+1 } N 
1 — e -{S+i)/2aM + as M — > oo, where c is a positive con- 
stant. This explains why as M increases, there is general 
trend that the period 2(2 — 1) dynamics orderly phase 
occurs less frequently. 

From Theorem [TJ the second longest period in the or- 
derly phase is of period 4(2 M ~ 1 - 1) for M > 2. This 
dynamics is associated with a characteristic polynomial 
in the form (A — l)p{x) where p{x) is a primitive poly- 
nomial of degree (M — 1). Again, using the idea in the 
proof of Theorem [JJ it is easy to check that the period 
4(2 M_1 — 1) orderly phase is quite stable (although it is 
not as stable as the period 2(2 M — 1) dynamics) as all 
but four conditional minority choice time series exhibit 
period one rather than period two dynamics. 

For those orderly phases with shorter periods, the num- 
ber of distinct histories present is small so that the effec- 
tive diversity of the strategies at play are greatly reduced. 
Combined with the presence of, in general, a larger pro- 
portion of period one conditional minority choice time 
series in this dynamics, these shorter period dynamics 
are much less stable in the sense that they can last for a 
much shorter time. 

The same analysis can be applied to the case when I > 
I C2 . In this case, the only oscillating agent in the system 
switches his strategy once every few turns. Applying the 
same analysis as in the proof of Theorem [JJ it can be 
shown that the maximum period of the orderly phase 
is 2 J (2 M — 1) where j is the number of turns between 
the adoption and termination of a strategy for the sole 
oscillating agent. In this situation, the strength of a lot of 
period two dynamics in the conditional minority choice 
time series are weakened making the dynamics in the 
orderly phase less pronounced. 

Note that one of the factors making the above inter- 
mittent behavior possible is that the number of agents 
N is odd. In case N is even, there are equal number of 
agents choosing each side. Hence, all agents are frozen 
and have 1/2 chance of correctly predicting the minority 
side. And the minority choice time series does not show 
any periodicity. This is exactly what we observe in our 
simulation. 

Let us discuss more about the agent cooperation. Even 
for a w c^f G and I sufficiently large, the average value of 
a 2 /N is still a little bit higher than the theoretical min- 
imum due to three reasons. First, a few initial quenched 
disorders does not allow maximum agent cooperation. 
Second, the system may be trapped in a non-maximally 
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cooperative state even though the initial quenched dis- 
order allows maximum agent cooperation. Third, as we 
have pointed out in Subsec. IIVBI the aperiodic phase 
of the intermittency is associated with the sub-maximal 
agent cooperation making the value of a 2 /N averaged 
over initial quenched disorder greater than the minimum 
value. So, it is not surprising to find that the values of 
a 2 /N for HMG MRSS and HMG FSS are about the same 
for a w af G and I > I Cl . 

2. The sub-case of I > I Cl and a < of ° 

For fixed M and S, the number of agents N increases 
as a decreases. From the discussions in Subsec. IIVBI we 
know for a sufficiently large N, the timescale for all ex- 
cept at most one of the agents to form crowd-anticrowd 
pairs is longer than the timescale for a pair of crowd- 
anticrowd agents to break up. Therefore, as a < ot~ , 
it is highly unlikely for the system to attain maximum 
agent cooperation. Besides, by further increasing a, over- 
crowding of strategies is severe as more and more agents 
are using the same strategy with similar virtual scores 
and number of turns since its adoption. As a result, mal- 
adaptation and herd effect begin to appear. These are 
the reasons why for I > I Cl , the value of a 2 /N starts 
to increase as a falls below about 0.1. This is also the 
reason why the minority choice time series conditioned 
on an arbitrary but fixed history exhibits a very weak 
period two dynamics. However this period two dynamics 
is much weaker than the one observed in the standard 
MG because the presence of inertia makes the agents to 
response less readily. As the effective strategy diversity 
of HMG FSS is greater than that of HMG MRSS which is 
in turn greater than that of the standard MG, the value 
of a 2 jN for HMG FSS is smaller than that of HMG MRSS 
which is in turn smaller than that of the standard MG 
provided that I > I Cl . 

Discussions in the previous two sub-cases predict that 
by decreasing a below w 2 2 /NS, the value of a 2 jN 
for HMG FSS will start to increase due to overcrowding of 
strategies. Unfortunately, we are not able to check the 
correctness of our prediction because the memory and 
run time requirements are too high. 

V. DISCUSSIONS 

In summary, we have performed extensive numerical 
simulations to study the behavior of HMG MRSS . We 
found that HMG agents cooperate better provided that 
their strategies are picked from the FSS instead of from 
the MRSS. This is because the effective diversity of 
strategies is higher in the former case. Based on the 
crowd-anticrowd theory [3, [H[ , we understood the ori- 
gin of agent cooperation for HMG MR in various pa- 
rameter ranges by studying the interplay between the 
so-called "crowd-anticrowd pair formation" driving force 



and the so-called "crowd-anticrowd pair destruction" 
driving force. And we found that the difference in the co- 
operative and dynamical behavior between HMG FSS and 
HMG MRSS is mainly due to the structure of the strategy 
space used. In particular, we were able to explain the 
novel intermittent behavior of the system and the orderly 
phase dynamics of the minority choice time series when 
a w af G , I > J Cl , N is odd and a 2 /N « 1/4JV. Es- 
sentially, this novel orderly dynamics in the intermittent 
phase is caused by the fact that the game is effectively 
reduced to a similar game played by only one agent most 
of the time and is accompanied by the maximum possible 
agent cooperation. And this reduction is possible in case 
the strategy pool is MRSS rather than FSS. 

On one hand, HMG MRSS appears to be special in the 
sense that it is the only variant of MG we have examined 
whose behavior depends sensitively on whether the FSS 
or the MRSS is used. On the other hand, the assump- 
tion that the behavior of standard MG when played in 
FSS or in MRSS is about the same is only a working as- 
sumption supported by numerical simulation results and 
heuristic arguments 0, . Although this assumption 
greatly simplifies the space complexity of numerical sim- 
ulations and sometimes even enable us to obtain a few 
semi- analytical results, one should bear in mind that this 
is only an assumption after all. In this regard, it is in- 
structive to study the conditions under which one can 
replace FSS by MRSS without significantly affecting the 
dynamics and cooperative behavior of a system. 

Lastly, we remark that the standard way of using the 
number 2 M+1 to measure the diversity of strategies is no 
longer appropriate for HMG. We believe that by suitably 
defining the diversity of strategies (and hence also the 
expression for a), the a 2 /N against a curves for HMG 
played using different choices of strategy spaces can be 
made to coincide, at least roughly. 
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APPENDIX A: PROOF OF CLAIM [T] 

Let T, T' denote two different partitions of the N play- 
ers into N/2 pairs. Let Prr denotes the probability that 
it is possible for each of the N/2 pairs of agents in the par- 
tition r to pick a sock of the same color but different side. 
Clearly, Prr's are exchangeable. That is, Prr = Prr/'. In 
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fact, for S<2 A/+1 , 

( (2 M + 1 )\ 



Prr 



2 S(Af+l)( 2 M+l _ £)] 



1-1- 



5 



2 Af+l 



2MNS 



(Al) 



By de Finetti theorem [20l| and its generalization by 
Diaconis and Freedman [21|, as long as < Prr < 1 for 
any finite N, the probability that there exists a way to 
form N/2 pairs of socks by picking one sock from each 
player is lower bounded by Pr(oo) — 0/N for some > 
independent of N, where Pr(oo) denotes the same prob- 
ability when N — ► oo. 

So, to prove this claim, it suffices to show that Pr(oo) = 
1 whenever 2 M+1 < NS and S > 2. Under these two con- 
ditions, it is obvious that Prr € (0, 1) for any finite N. 
So, de Finetti theorem implies that Prp's are condition- 
ally independent given the tail cr-field 1201. Thus, 



Pr(oo) = lim 1 - TT(1 - Pr r ) = 1 



Hence, claim [TJ is proved. 



(A2) 



□ 



Before leaving this appendix, we point out a common 
mistake people makes. The probability that all paired 
players have matching socks for an arbitrary but fixed 

r qi n/2 

partition T is equal to P(r) « 1 - (l - S/2 M+1 ) 

Thus, the expected number of partitions satisfying this 
"all paired players" condition is P(r) x 7V!/[(7V/2)!] 2 . A 
threshold condition for the existence of a partition that 
all paired players have matching socks can then be de- 
duced by saddle point approximation. The loophole in 
this argument is that socks are assigned once and for all 
to the players so that the probabilities P(r) and P(r') 
are not independent. 



APPENDIX B: PROOF OF THEOREM [TJ 

This proof uses a number of finite field concepts and 
techniques. Readers who are not familiar may consult 
Ref. [22j before moving on. The characteristic equation 
of Eq. ® is 



M 



. 



(Bl) 



i=i 



If the characteristic equation has no degenerate root, 
the homogeneous part of the solution of \j, n in Eq. ([6]) 
is in the form J2i 9i w f where Wi are roots of Eq. (|B1|) 
over the extension field GF(2 M ) for some g t E GF(2 M ). 
Clearly, the period of the sequence {J2i 9i w ?}t=o divides 
L.C.M.(|ii>i|, \tV2\, ■ ■ ■ , \ivm\)j where |iUj| denotes the or- 
der of Wi. And the equality holds if g^s are all non-zero. 
(Remember also that \wi\ divides (2 M — 1) for all i.) 



What if the roots of Eq. (|B1[) are degenerate? Sup- 
pose u>i is a double root of Eq. (|B1|) . Then, and nro™ 
are the two generators of the homogeneous part of the 
solution of Eq. ©. Besides, the period of the sequence 
{giw± + g2nwi} divides 2|toi| with the equality holds if 
ffi, ff2 7^ 0. Similarly, let 2-? be the smallest integer greater 
than or equal to k. Then, it is straight-forward to check 
that 2 J |u>i| is divisible by the period of the homogeneous 
part of the solution of the recursion relation ([6]) that cor- 
responds to a degree k root Wi of the characteristic equa- 
tion (|Bip . Furthermore, the period can attain the value 
2 J |wi| provided that the coefficients g^s are non-zero. 

Combining the two cases above, we conclude that the 
period of the homogeneous solution of Eq. ([6]) divides 
2 J 'L.C.M.(|t0i|, 1 102 1, ■ ■ ■) where w^s are the distinct roots 
of its characteristic equation and j is a non-negative inte- 
ger. The maximum possible period for this homogeneous 
solution is (2 M — 1); and this is attainable provided that 
the coefficient in each term of the homogeneous solution 
is non-zero and the characteristic polynomial of Eq. ^ 
is any one of the f{2 M — 1)/M primitive polynomials of 
degree M in the GF(2)[X], where ip is the Euler-Totient 
function. Note further that if the minority choice time 
series were generated by the homogeneous solution of 
Eq. © alone, then this time series would not contain M 
consecutive 0's should its period be (2 M — 1). Besides, 
in one period, all history strings but the one consists of 
all 0's would appear exactly once. 

Now, we study the particular solution of Eq. ©. The 
first case to consider is when the characteristic polyno- 
mial has an odd number of (non-zero) terms. (This in- 
cludes the case when the polynomial is irreducible and 
M > 2 for otherwise 1 is a root of Eq. (|B1[) .) In this case, 
a particular solution of Eq. ^ is fi n = r/o + n (mod 2) 
if \{i : r\ n -2i ^ 0}| is even or \x n = rja + n + 1 (mod 2) if 
\{i : T) n ~2i 7^ 0}| is odd. In either case, the period of this 
particular solution is 2. 

The remaining case is when the number of (non-zero) 
terms in Eq. (|F31|) is even. In this case, (A — 1) must be 
a factor of the R.H.S. of Eq. jBl]). We write the R.H.S. 
of Eq. (|B1[) as (A — l)' c p(A) for some polynomial p(X) 
with p(l) =/= 0. If k = 1, a particular solution is fi n = 
|_(n + 1 — rjo)/2\ (mod 2) which is of period 4. Similarly, 
it is easy to check that there is a particular solution of 
Eq. © whose period divides 2 J where 2 J is the smallest 
integer greater than [k + 1). 

By combining the studies of the periods of the homo- 
geneous and particular solutions of Eq. ^ , we conclude 
that the period of the general solution of Eq. ([6]) is in the 
form 2- 7 L.C.M.(|wi |, [ws], . . .) where lUj's are the distinct 
roots of Eq. (|Bip and 2 J is the smallest integer greater 
than (fc + 1). Moreover, the maximum possible period 
for the solution of the recursion relation ([6]) is 2(2 M — 1) 
for M > 2 and 4 for M = 1. In the former case, such 
a maximum period is attained only if the characteristic 
polynomial of Eq. ([B]) is primitive. And in the latter case, 
it is attained when the characteristic polynomial is A — 1. 

Finally, we consider the case when the period of the 
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sequence {/x n } attains its maximum possible value of 
2(2 M — 1). In this case, a particular solution of Eq. © 
corresponds to the histories of alternating O's and l's. 
So, combined with our earlier analysis on the frequency 
of histories for the homogeneous part of the solution of 



Eq. ([6]), we conclude that in each period, all but the two 
histories of alternating O's and l's appear twice. Besides, 
the two histories of alternating O's and l's appear once. 
This proves the theorem. □ 
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